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1 CONTROLLING FLOW OF DATA BETWEEN DATA PROCESSING SYSTEMS VIA 

2 A MEMORY 

3 CROSS REFERENCE 

4 This application is cross referenced with Docket IL920000076US1 , having the same title 

5 and being filed on the same day. The cross referenced docket is here included by 

6 reference in entirety for all purposes. 

7 FIELD OF INVENTION 

8 The present invention relates to a method and apparatus for controlling flow of data via a 

9 memory between first and second data processing systems such as a host computer 

1 0 system and a data communications interface for conmiunicating data between the host 

1 1 computer system and a data communications network. 

12 BACKGROUND OF THE INVENTION 

13 A conventional data processing network comprises a plurality of host computer systems 

14 and a plurality of attached devices all interconnected by an intervening network 

1 5 architecture such as an Ethernet architecture. The network architecture typically 

16 comprises one or more data communications switches. The host computer systems and 

1 7 the attached devices each form a node in the data processing network. Each host 
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1 computer system typically comprises a plurality of central processing units and data 

2 storage memory device interconnected by a bus architecture such as a PCI bus 

3 architecture. A network adapter is also connected to the bus architecture for 

4 communicating data between the host computer system and other nodes in the data 

5 processing network via the network architecture. It would be desirable for transfer of data 

6 and control information between the host computer system and the network architecture 

7 to be facilitated as efficiently as possible. 

8 SUMMARY OF THE INVENTION 

9 In one aspect of the present invention, there is now provided apparatus for controlling 

1 0 flow of data between first and second data processing systems via a memory, the 

1 1 apparatus comprising descriptor logic for generating a plurality of descriptors including a 

1 2 fi'ame descriptor defining a data packet to be communicated between a location in the 

13 memory and the second data processing system, and a pointer descriptor identifying the 

14 location in the memory; and a descriptor table for storing the descriptors generated by the 

1 5 descriptor logic for access by the first and second data processing systems. The 

1 6 descriptor logic and descriptor table improve efficiency of data flow control between the 

1 7 first and second data processing systems such as a host computer system and a data 

1 8 communications interface for communicating data between the host computer system and 

19 a data communications network. The descriptor table may be stored in the memory of the 

20 host computer system. Alternatively, the descriptor table is stored in a memory of the data 

21 communications interface. The descriptor logic may also generate a branch descriptor 

22 comprising a link to another descriptor in the descriptor table. The descriptor table may 

23 comprises a plurality of descriptor lists sequentially linked together via branch descriptors 

24 therein. Alternatively, the descriptor table may comprise a cyclic descriptor list. 
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1 Another aspect of the present invention extends to a data processing system extends to a 

2 data processing system comprising a host processing system having a memory, a data 

3 communications interface for communicating data between the host computer system and 

4 a data communications network, and apparatus as hereinbefore described for controlUng 

5 flow of data between the memory of the host computer system and the data 

6 communications interface 

7 Viewing the present invention from another aspect, there is now provided a method for 

8 controlling flow of data between first and second data processing systems via a memory, 

9 the method comprising: by descriptor logic, generating a plurality of descriptors including 

10 a frame descriptor defining a data packet to be communicated between a location in the 

1 1 memory and the second data processing system, and a pointer descriptor identifying the 

12 location in the memory; and storing the descriptors generated by the descriptor logic in a 

1 3 descriptor table for access by the first and second data processing systems. 

14 BRIEF DESCRIPTION OF THE DRAWINGS 

1 5 These and other objects, features, and advantages of the present invention will become 

16 apparent upon fiirther consideration of the following detailed description of the invention, 

17 by way of example only, when read in conjunction with the drawing figures, in which: 

1 8 Fig. 1 is a block diagram of an example of a data processing network; 

1 9 Fig. 2 is a block diagram of a network interface adapter card for the data processing 

20 network; 

21 Fig, 3 is a block diagram of an example of a host computer system for the data network; 



Docket Number IL920000077US1 



-3- 



1 Fig. 4 is a block diagram of an example of an Integrated System on a Chip (ISOC) for the 

2 network adapter card; 

3 Fig. 5 is another block diagram of the ISOC; 

4 Fig. 6 is a block diagram of the ISOC demonstrating information flow through the ISOC ; 

5 Fig. 7 is a block diagram of a logical transmit path through the ISOC; 

6 Fig. 8 is a block diagram of a logical receive path through the ISOC; 

7 Fig. 9A is a block diagram of a cyclic descriptor table 

8 Fig. 9B is a block diagram of a linked set of descriptor tables; 

9 Fig. 10 is a block diagram of a virtual buffer and its physical counterpart buffer; 

10 Fig. 1 1 is a block diagram of a completion queue; 

1 1 Fig. 12 is a block diagram of a transmit flow of data from the host to the network; 

12 Fig. 13 is another block diagram of a transmit flow of data from the host to the network; 

1 3 Fig. 14 is a block diagram of a receive flow of data from the network to the host; and 

14 Fig. 1 5 is another block diagram of a receive flow of data from the network to the host. 
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DESCRIPTION OF THE INVENTION 



2 The present invention provides methods, systems and apparatus for controlling flow of 

3 data between first and second data processing systems via a memory, the apparatus 

4 comprising descriptor logic for generating a plurality of descriptors including a frame 

5 descriptor defining a data packet to be communicated between a location in the memory 

6 and the second data processing system, and a pointer descriptor identifying the location in 

7 the memory; and a descriptor table for storing the descriptors generated by the descriptor 

8 logic for access by the first and second data processing systems. The descriptor logic and 

9 descriptor table improve efficiency of data flow control between the first and second data 

10 processing systems such as a host computer system and a data communications interface 

1 1 for communicating data between the host computer system and a data communications 

1 2 network. The descriptor table may be stored in the memory of the host computer system. 

1 3 Altematively, the descriptor table is stored in a memory of the data communications 

14 interface. The descriptor logic may also generate a branch descriptor comprising a link to 

15 another descriptor in the descriptor table. The descriptor table may comprises a plurality 

16 of descriptor lists sequentially linked together via branch descriptors therein. 

1 7 Altematively, the descriptor table may comprise a cyclic descriptor list. 

18 In more particular embodiments, the present invention extends to a data processing 

19 system comprising a host processing system having a memory, a data communications 

20 interface for communicating data between the host computer system and a data 

21 communications network, and apparatus as hereinbefore described for controlling flow of 

22 data between the memory of the host computer system and the data conmiunications 

23 interface 

24 There is now also provided a method for controlling flow of data between first and 

25 second data processing systems via a memory, the method comprising: by descriptor 

26 logic, generating a plurality of descriptors including a frame descriptor defining a data 
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1 packet to be communicated between a location in the memory and the second data 

2 processing system, and a pointer descriptor identifying the location in the memory; and 

3 storing the descriptors generated by the descriptor logic in a descriptor table for access by 

4 the first and second data processing systems. 

5 Referring first to Figure 1 , an example of a data processing network embodying the 

6 present invention comprises a plurality of host computer systems 10 and a plurality of 

7 attached devices 20 interconnected by an intervening network architecture 30 such as an 

8 InfiniBand network architecture (InfiniBand is a trade mark of the InfiniBand Trade 

9 Association). The network architecture 30 typically comprises a plurality of data 

10 communications switches 40. The host computer systems 10 and the attached devices 20 

1 1 each form a node in the data processing network. Each host computer system 1 0 

1 2 comprises a plurality of central processing units (CPUs) 50, and a memory 60 

1 3 interconnected by a bus architecture 70 such as a PCI bus architecture. A network adapter 

14 80 is also connected to the bus architecture for conmiunicating data between the host 

1 5 computer system 1 0 and other nodes in the data processing network via the network 

16 architecture 30. 

1 7 Referring now to Figure 2, in particularly preferred embodiments of the present invention, 

1 8 the network adapter 80 comprises a pluggable option card having a connector such as an 

1 9 edge connector for removable insertion into the bus architecture 70 of the host computer 

20 system 10. The option card carries an Application Specific Integrated Circuit (ASIC) or 

21 Integrated System on a Chip (ISOC) 120 connectable to the bus architecture 70 via the 

22 connector 1 70, one or more third level memory modules 250 connected to the ISOC 1 20, 

23 and an interposeer 260 connected to the ISOC 120 for communicating data between the 

24 media of the network architecture 30 and the ISOC 120. The interposer 260 provides a 

25 physical connection to the network 30. In some embodiments of the present invention, the 

26 interposer 260 may be implemented in a single ASIC. However, in other embodiments of 

27 the present invention, the interposer 260 may be implemented by multiple components. 
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1 For example, if the network 30 comprises an optical network, the interposer 260 may 

2 comprise a retimer driving a separate optical transceiver. The memory 250 may be 

3 implemented by SRAM, SDRAM, or a combination thereof. Other forms of memory may 

4 also be employed in the implementation of memory 250. The ISOC 120 includes a first 

5 and a second memory. The memory subsystem of the adapter 80 will be described shortly. 

6 As will become apparent from the following description, this arrangement provides: 

7 improved performance of distributed applications operating on the data processing 

8 network; improved system scaleability; compatibility with a range of communication 

9 protocols; and reduced processing requirements in the host computer system. More 

10 specifically, this arrangement permits coexistence of heterogeneous communication 

1 1 protocols between the adapters 80 and the host systems 10. Such protocols can serve 

1 2 various applications, use the same adapter 80, and use a predefined set of data structures 

1 3 thereby enhancing data transfers between the host and the adapter 80. The number of 

14 application channels that can be opened in parallel is determined by the amount of 

1 5 memory resources allocated to the adapter 80 and is independent of processing power 

16 embedded in the adapter. It will be appreciated fi-om the following that the ISOC 120 

1 7 concept of integrating multiple components into a single integrated circuit chip 

1 8 component advantageously minimizes manufacturing costs and in provides reusable 

19 system building blocks. However, it will also be appreciated that in other embodiments of 

20 the present invention, the elements of the ISOC 1 20 may be implemented by discrete 

21 components. 

22 In the following description, the term Frame refers to data units or messages transferred 

23 between software running on the host computer system 10 and the adapter 80. Each 

24 Frame comprises a Frame Header and a data payload. The data payload may contain user 

25 data, high level protocol header data, acknowledgments, flow control or any combination 

26 thereof. The contents of the Frame Header will be described in detail shortly. The adapter 

27 80 processes only the Frame Header. The adapter 80 may fi-agment Frames into smaller 
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1 packets which are more efficiently transported on the network architecture 30. However, 

2 such fragmentation generally does not transform the data payload. 

3 In particularly preferred embodiment of the present invention, data is transported on the 

4 network architecture 30 in atomic units hereinafter referred to as Packets. Each Packet 

5 comprises route information followed by hardware header data and payload data. In a 

6 typical example of the present invention, a packet size of up to 1024 bytes is employed. 

7 Frames of larger size are fragmented into 1024 byte packets. It will be appreciated that in 

8 other embodiments of the present invention, different packet sizes may be employed. 

9 In a preferred embodiment of the present invention, communications between the adapter 

10 80 and multiple applications running on the host computer system 1 0 are effected via a 

1 1 Logical Communication Port architecture (LCP). The adapter 80 comprises a memory 

12 hierarchy which allows optimization of access latency to different intemal data structures. 

1 3 This memory hierarchy will be described shortly. In preferred embodiments of the present 

14 invention, the adapter 80 provides separate paths for outbound (TX) data destined for the 

15 network architecture 30 and inbound (RX) data destined for the host computer system 10. 

16 Each path includes it own data transfer engine, header processing logic and network 

1 7 architecture interface. These paths will alas be described in detail shortly. 

1 8 Referring now to Figure 3, the LCP architecture defines a framework for the interface 

19 between local consumers running on the host computer system 10 and the adapter 80. 

20 Examples of such consumers include both applications and threads. The computer system 

21 10 can be subdivided into a user application space 90 and a kernel space 1 10. The LCP 

22 architecture provides each consumer with a logical port into the network architecture 30. 

23 This port can be accessed directly from a user space 90. In particularly preferred 

24 embodiments of the present invention, a hardware protection mechanism takes care of 

25 access permission. An LCP registration is performed by in the kernel space 1 10 prior to 

26 transfer of data frames. The LCP architecture need not define a communication protocol. 
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1 Rather, it defines an interface between the appHcations and the adapter 80 for transfer of 

2 data and control information. Communication protocol details may be instead set by the 

3 application and program code executing in the adapter 80. The number of channels that 

4 can be used on the adapter 80 is limited only by the amount of memory on the adapter 

5 card 80 available for LCP related information. Each LCP port can be programmable to 

6 have a specific set of features. The set of features is selected according to the specific 

7 protocol to best support data transfer between the memory 60 in the host computer system 

8 and the adapter 80. Various communication protocols can be supported simultaneously, 

9 with each protocol using a different LCP port. 

10 The LCP architecture comprises LCP Clients 100, an LCP Manager 130 resident in the 

1 1 kernel space 130, and one or more LCP Contexts 140 resident in the adapter 80. 

12 Each LCP Client 100 is a unidirectional application end point connected to an LCP port. 

13 An LCP client 100 can be located in the user application space 90 or in the kemel 1 10. In 

14 operation, each LCP client 100 produces commands and data to be read from the memory 

15 60 and transferred by the adapter 80 via a TX LCP channel, or consumes data transferred 

16 by the adapter 80 to the memory 60 via an RX LCP channel. 

1 7 The LCP Manager 1 30 is a trusted component that services request for LCP channel 

1 8 allocations and deallocations and for registration of read/write areas in the memory 60 for 

19 each channel. The LCP Manager 130 allows a user space application to use resources of 

20 the adapter 80 without compromising other communication operations, applications, or 

21 the operating system of the host computer system 10. 

22 Each LCP Context 140 is the set of control information required by the adapter 80 to 

23 service a specific LCP Client 100. The LCP Context 140 may include LCP channel 

24 attributes which are constant throughout existence of the channel, such as possible 

25 commands, pointer structure, and buffer descriptor definitions. The LCP Context 140 
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1 may also include specific LCP service information for the LCP channel, such as the 

2 amount of data waiting for service, and the next address to access for the related LCP 

3 channel. The LCP context 140 is stored in memory resident in the adapter 80 to enable 

4 fast LCP context switching when the adapter 80 stops servicing one channel and starts 

5 servicing another channel. 

6 An LCP Client 1 00 requiring initiation of an LCP port turns to the LCP Manager 1 30 and 

7 requests the allocation of an LCP channel. The LCP channel attributes are determined at 

8 this time and prescribe the behavior of the LCP port and the operations that the LCP 

9 Client 100 is authorized to perform in association with the LCP port. The LCP Client 100 

10 is granted an address that will be used to access the adapter 80 in a unique and secure 

1 1 way. This address is known as a Doorbell Address. 

1 2 The LCP Manager 1 30 is also responsible for registering areas of the host memory 60 to 

1 3 enable virtual to physical address translation by the adapter, and to allow user space 

14 clients to access these host memory areas without tampering with other programs. 

1 5 Registration of new buffers and deregistration of previous buffers can be requested by 

1 6 each LCP Client 1 00 during run-time. Such a change, requires a sequence of information 

17 exchanges between the LCP Client 100, the LCP Manager 130, and the adapter 80. 

18 Each LCP Client 100 and port are associated with an LCP Context 140 that provides all 

1 9 the information required by the adapter 80 to service pending requests sent by the LCP 

20 port for command execution. 

21 To initiate memory transfers between the LCP Client 100 and the adapter 80, and initiate 

22 transmission of frames, the LCP Client 100 prepares descriptors holding the information 

23 for a specific operation. The LCP Client 100 then performs an I/O write to the Doorbell 

24 address mapped to the adapter 80. Writing to the Doorbell address updates the LCP 
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1 Context 140 on the adapter 80, adding the new request for execution. The adapter 80 

2 arbitrates between various transmit LCP ports that have pending requests, and selects the 

3 next one to be serviced. 

4 On receipt of data, the Frame and LCP for a received packet are identified. Descriptors 

5 are generated to define the operation required for the receive LCP. Execution of these 

6 descriptors by an LCP Engine of the adapter 80 stores the incoming data in an appropriate 

7 data buffer allocated to the LCP channel in the memory 60 of the host computer system 

8 1 0. For each LCP channel serviced, the adapter 80 loads the associated LCP context 

9 information and uses this information to perform the desired set of data transfers. The 

10 adapter 80 then continues on to process the next selected LCP Context 140. 

1 1 Referring now to Figure 3, and as mentioned earlier, the ISOC 120 comprises a first 

1 2 memory space 220 and 230 and a second memory space 240 and the adapter 80 fiirther 

1 3 comprises a third level memory 250. The first, second, and third memory spaces for part 

14 of a memory subsystem 2 1 0 of the adapter 80. In a preferred embodiment of the present 

1 5 invention, the ISOC 120 comprises a TX processor (TX MPC) 1 50 dedicated to data 

1 6 transmission operations and an RX processor (RX MPC) 1 60 dedicated to data reception 

1 7 operation. In particularly preferred embodiments of the present invention, processors 150 

18 and 1 60 are implemented by Reduced histruction Set Computing (RISC) microprocessors 

1 9 such as IBM PowerPC 405 RISC microprocessors. Within the memory subsystem 2 1 0, 

20 the ISOC 1 20 comprises, in addition to the first and second memory spaces, a data cache 

21 1 80 and an instruction cache 1 70 associated with TX processor 150, together with a 

22 second data cache 1 90 and second instruction cache 190 associated with RX processor 

23 1 60. The difference between the three levels is the size of memory and the associated 

24 access time. As will become apparent shortly, the memory subsystem 2 1 0 facilitates: 

25 convenient access to instruction and data by both the TX processor 1 50 and the RX 

26 processor 1 60; scaleability; and sharing of resources between the TX processor 1 50 and 

27 the RX processor 1 60 in the interests of reducing manufacturing costs. 



Docket Number IL920000077US1 



-11- 



1 The first level memory spaces (Ml ) 220 and 230 comprise a TX-Ml memory space 220 

2 and RX-Ml memory space 230. The TX-Ml memory 220 can be accessed only by the 

3 TX processor 1 50 and the RX-Ml memory 230 can be accessed only by the RX processor 

4 160. In operation the first level memory spaces 220 and 230 are used to hold temporary 

5 data structures, header templates, stacks, etc. The first level memory spaces 220 and 230 

6 both react to zero wait states. Each one of the first level memory spaces 220 and 230 is 

7 connected only to the data interface of the corresponding one of the processors 1 50 and 

8 1 60 and not to the instruction interface. This arrangement enables both cacheable and 

9 non-cacheable first level memory areas available while maintaining efficient access to 

1 0 data in the first level memory spaces 230 and 240. 

1 1 The second level memory space (M2) 240 is a shared memory available to both 

12 processors 150 and 160, other components of the adapter 80, and to the host computer 

1 3 system 1 0. Access to the second level memory space 240 is slower than access to the first 

14 level memory areas 220 and 230 because the second level memory space 240 is used by 

1 5 more agent via a shared intemal bus. The third level memory space 250 is also a shared 

1 6 resource. In particularly preferred embodiments of the present invention the adapter 80 

1 7 comprises a computer peripheral circuit card on which the first level memory spaces 220 

1 8 and 230 and the second level memory space 240 are both integrated on the same ASIC as 

1 9 the processors 1 50 and 1 60. The shared memory spaces 240 and 250 are generally used 

20 for data types that do not require fast and frequent access cycles. Such data types include 

21 LCP contexts 140 and virtual address translation tables. The shared memory spaces 240 

22 and 250 are accessible to both instruction and data interfaces of the processors 1 50 and 

23 160. 

24 The adapter 80 handles transmission and reception data flows separately. The separate 

25 processor 1 50 and 1 60 for the transmission and reception path avoids the overhead of 

26 switching between task, isolates temporary processing loads in one path fi-om the other 
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1 path, and facilitates use of two embedded processors to process incoming and outgoing 

2 data streams. Referring now to Figure 5, the ISOC 120 comprises transmission path logic 

3 280 and reception path logic 290, and shared logic 300. The transmission path logic 280 

4 comprises an LCP TX engine 3 10 for decoding specifics of each LCP channel and 

5 fetching LCP related commands for execution; TX logic 320 for controlling transfer of 

6 frames into the adapter 80, the aforementioned TX processor 150 for managing TX frame 

7 and packet processing; the aforementioned first level TX memory 220 for holding 

8 instructions and temporary data structures; and link logic 330; and logic for assisting the 

9 TX processor 1 50 in managing the data flow and packet processing such as routing 

10 processing for fragmentation of frames into data packets. The TX processor 150 

1 1 processes tasks in series based on a polling only scheme in which the processor is 

12 interrupted only on exceptions and errors. The first level TX memory 220 is employed by 

1 3 the processor 1 50 for conmiunicating with TX logic 320. The reception path logic 290 

14 comprises link logic 340; hardware for assisting the aforementioned RX processor 160 in 

1 5 processing headers of incoming packets and transformation or assembly of such packets 

16 into frames; the aforementioned RX processor 160 for RX frame and packet processing; 

17 the aforementioned first level RX memory 230 for holding instructions; RX logic 350 for 

1 8 controlling transfer of frames from the network architecture 30; and an LCP RX engine 

19 360 for decoding the specifics of each LCP channel, storing the incoming data in the 

20 related LCP data structures in the memory 60 of the host computer system, and accepting 

21 and registering pointers to empty frame buffers as they are provided by the LCP Client 

22 1 00 for use by the adapter 80. The RX processor 1 60 processes tasks in series using a 

23 polling only scheme in which the RX processor 1 60 is interrupted only on exceptions or 

24 errors. The level 1 RX memory 230 is used by the RX processor 160 to communicate 

25 with the RX logic 350. 

26 As mentioned earlier, the ISOC approach permits reduction in manufacturing costs 

27 associated with the adapter 80 and the other components thereof, such as the circuit board 

28 and the other supporting modules. The ISOC approach also increases simplicity of the 
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1 adapter 80, thereby increasing reliability. The number of connections between elements of 

2 the ISOC 120 is effectively unlimited. Therefore, multiple and wide interconnect paths 

3 can be implemented. In the interests of reducing data processing overheads in the host 

4 computer system 10, data transfer operations to and from the host memory 60 are 

5 predominantly performed by the ISOC 120. The ISOC 120 also performs processing of 

6 the header of incoming and outgoing packets. During transmission, the ISOC 1 20 builds 

7 the header and routes it to the network architecture 30. During reception, the adapter 80 

8 processes the header in order to determine its location in the system's memory. The level 

9 1 memories 220 and 230 are zero wait state memories providing processor data space 

10 such as stack, templates, tables, and temporary storage locations. In especially preferred 

1 1 embodiments of the present invention, the transmission path logic 280, reception path 

12 logic 290, and shared logic 300 are built from smaller logic elements referred to as cores. 

1 3 The term core is used because there elements are designed as individual pieces of logic 

1 4 which have stand-alone properties enabling them to be used for different applications. 

1 5 As indicated earlier, the transmission path logic 280 is responsible for processing 

1 6 transmission or outgoing frames. Frame transmission is initiated via the bus architecture 

17 70 by a CPU such as CPU 50 of the host computer system 10. The ISOC 120 comprises 

18 bus interface logic 370 for communicating with the bus architecture 70. The ISOC 120 

1 9 also comprises bus bridging logic 390 connecting the bus interface logic 370 to a 

20 processor local bus (PLB) 390 of the ISOC 120. The TX LCP engine 310 fetches 

21 commands and frames from the host memory 60. The TX processor 150 processes the 

22 header of each frame into a format suitable for transmission as packets on the network 

23 architecture 30. The TX logic 320 transfer the frame data without modification. The link 

24 logic 330 processes each packet to be transmitted into a final form for transmission on the 

25 network architecture 30. The link logic 330 may comprises one or more ports each 

26 connectable to the network architecture 30. 
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1 As indicated earlier, the reception path logic 290 is responsible for processing incoming 

2 packets. Initially, packets received from the network architecture 30 are processed by link 

3 logic 340. Link logic 340 recreates the packet in a header and payload format. To 

4 determine the packet format and its destination in the host memory 60, the header is 

5 processing by the RX processor 230. The link logic 340 may comprises one or more ports 

6 each connectable to the network architecture 30. The RX LCP engine is responsible for 

7 transferring the data into tiie host memory 60 via the bus architecture 70. 

8 The transmission path logic 280 comprises a Headerin first in- first out memory (FIFO) 

9 400 between the TX LCP engine 3 1 0 and the TX processor 220. The reception path logic 

10 comprises a HeaderOut FIFO 410 between the RX processor 230 and the RX LCP engine 

1 1 360. Additional FIFOs and queues are provided in the TX logic 320 and the RX logic 

12 350. These FIFOs and queues will be described shortly. 

1 3 The shared logic 300 comprises all logical elements shared by the transmission path logic 

14 280 and the reception path logic 290. These elements include the aforementioned bus 

1 5 interface logic 370, bus bridging logic 380, PLB 390, second level memory 240 and a 

1 6 controller 420 for providing access to the remote third level memory 250. The bus 

1 7 interface logic 370 operates as both master and slave on the bus architecture 70. As a 

1 8 slave, the bus interface logic allows the CPU 50 to access the second level memory 240, 

1 9 the third level memory 250 via the controller 420, and also configuration registers and 

20 status registers of the ISOC 120. Such registers can generally be accessed by the CPU 50, 

21 the TX processor 1 50 and the RX processor 1 60. As a master, the bus interface logic 

22 allows the TX LCP engine 3 1 0 and the RX LCP engine 360 to access the memory 60 of 

23 the host computer system 1 0. hi Figure 5, "M" denotes a master connection and "S" 

24 denotes a slave connection. 

25 Referring now to Figure 6, packet flow through the ISOC 1 20 is generally symmetrical. In 

26 other words, the general structure of flow is similar in both transmit and receive 
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1 directions. The ISOC 120 can be regarded as comprising first interface logic 440; a first 

2 control logic 460; processor logic 480; second control logic 470; and second interface 

3 logic 450. Packets are processed in the following manner: 

4 A. In the transmit direction, information is brought into the ISOC 1 20 from the bus 

5 architecture 70 through the first interface logic. In the receive direction, 

6 information is brought into the ISOC 1 20 from the network architecture 30 

7 through the second interface logic 450. 

8 B. In the transmit direction, information brought into the ISOC 1 20 through the first 

9 interface logic 440 is processed by the first control logic 460. hi the receive 

10 direction, information brought into the ISOC through the second interface logic 

1 1 450 is processed by the second control logic 470, 

12 C. In the transmit direction, a frame header is extracted for an outgoing frame at the 

13 first control logic 460 and processed by the processor logic 480. The processor 

14 logic 480 generates instructions for the second control logic 470 based on the 

1 5 frame header. The payload of the outgoing frame is passed to the second interface 

1 6 logic 470. In the receive direction, a frame header is extracted from an incoming 

1 7 frame at the second control logic 470 and processed by the processor logic 480. 

1 8 The processor logic 480 generates instructions for the first control logic 460 based 

1 9 on the frame header. The payload of the incoming frame is passed to the first 

20 control logic 460. In both directions, the processor 480 is not directly handUng 

21 payload data. 

22 D. In the transmit direction, the second control logic 470 packages the outgoing 

23 payload data according to the instructions received from the processor logic 480. 

24 In the receive direction, the first control logic 460 packages the incoming payload 

25 according to the instructions received from the processor logic 480. 
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1 E. In the transmit direction, the information is moved through the second interface 

2 logic 450 to its destination via the network architecture 30. In the receive 

3 direction, the information is moved through the first interface logic to its 

4 destination via the bus architecture 70. 

5 An interface to software operating on the host computer system 10 is shown at 430. 

6 Similarly, interfaces to microcode operating on the processor inputs and outputs is shown 

7 at 490 and 500. 

8 Referring to Figure 7, what follows now is a more detailed description of one example of 

9 a flow of transmit data frames through the ISOC 120. The ISOC 120 can be divided into 

1 0 an LCP context domain 5 1 0, a frame domain 520 and a network domain 530 based on the 

1 1 various foraiats of information within the ISOC 1 20. The TX LCP engine 3 1 0 comprises 

12 an LCP requests FIFO 550, Direct Memory Access (DMA) logic 560, frame logic 580, 

13 and the aforementioned LCP context logic 140. The LCP request FIFO 550, DMA logic 

14 560, and LCP TX Context logic 590 reside in the LCP context domain 5 1 0. The frame 

1 5 logic 580 resides in the frame domain 520. The TX logic 320, first level TX memory 

1 6 space 220, and TX processor 1 50 straddle the boundary between the frame domain 520 

17 and the network domain 530. The TX link logic 330 resides in the network domain 530. 

1 8 In particularly preferred embodiments of the present invention, the Headerin FIFO 400 is 

19 integral to the first level TX memory space 220. In general, an application executing on 

20 the host computer system 10 creates a frame. The frame is then transmitted using a TX 

21 LCP channel on the adapter 80. Handshaking between the application and the adapter 80 

22 assumes a prior initialization performed by the LCP Manager 1 30. To add an LCP 

23 Service Request, an LCP Client 1 00 informs the adapter 80 that one or more additional 

24 transmit frames are ready to be executed. This is performed by writing to a control word 

25 in to a Doorbell The Doorbell's addresses are allocated in such as way that the write 

26 operation is translated into a physical write cycle on the bus architecture 70, using an 
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1 address that is uniquely associated with the LCP port and protected from access by other 

2 processes. The adapter 80 detects the write operation and logs the new request by 

3 incrementing an entry of previous requests for the specific LCP Client 1 00. This is part of 

4 the related LCP Context 140. An arbitration list, retained in the memory subsystem 210 

5 of the adapter 80 is also updated. In a simple example, arbitration uses the 

6 aforementioned FIFO scheme 550 between all transmit LCP channels having pending 

7 requests. While one LCP channel is serviced, the next LCP chaimel is selected. The 

8 service cycle begins when the corresponding LCP Context is loaded into the TX LCP 

9 engine 3 1 0. The LCP Context 140 is then accessed to derive atomic operations for 

10 servicing the LCP channel and to determine parameters for such operations. For example, 

1 1 such atomic operations may be based on LCP channel attributes recorded in the LCP 

12 Context 140. A complete service cycle typically includes a set of activities performed by 

13 the adapter 80 to fetch and execute a plurality of atomic descriptors created by the LCP 

14 Client 100. In the case of a TX LCP channel, the service cycle generally includes reading 

1 5 multiple frames from the host memory 60 into the memory subsystem 2 1 0 of the adapter 

16 80. Upon conclusion, all the LCP Context information requiring modification (in other 

17 words, the LCP Service Information) is updated in the memory subsystem 210 of the 

1 8 adapter 80. In general, the first action performed by the adapter 80 within the LCP 

19 Service cycle, is to fetch the next descriptor to be processed. 

20 Processing of transmission frames by the ISOC 120 typically includes the following 

21 steps: 

22 A. Fetching the subsequent LCP port frame descriptor. 
23 

24 The address of the next descriptor to be fetched is stored as parts of the LCP 

25 channel's Context 140. The adapter 80 reads the descriptor from host memory 60 

26 and decodes the descriptor based on the LCP channel attributes. The descriptor 
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1 defines the size of the new frame header, the size of the data payload, and the 

2 location of these items. 

3 B. Conversion of virtual address to physical address. 

4 If a data buffer is referenced by virtual memory addresses in an application, the 

5 address should go through an additional process of address translation, hi this 

6 case, the virtual address used by the application is translated into a physical 

7 address usable by the adapter 80 while it access the host memory 60. This is done 

8 by monitoring page boundary crossings and using physical page location 

9 information written by the LCP manager 1 30 into the memory subsystem 2 10 of 

10 the adapter 80. The virtual to physical translation process serves also as a security 

1 1 measure in cases where a descriptor table is created by an LCP client 100 which is 

12 not trusted. This prevents unauthorized access to unrelated areas of the host 

13 memory 60. 

1 4 C. Reading the frame header. 

1 5 Using physical addressing, the header and payload data of the TX frame are read 

1 6 from buffers in the host memory 60. The header is then stored in the TX Headerin 

1 7 FIFO 400. When the header fetch is completed, the adapter 80 sets an intemal flag 

1 8 indicating that processing of the header can be initiated by the TX processor 1 50. 

19 D. Reading the frame data. 

20 The payload data is read from the host memory 60 and stored by the adapter 80 in 

21 a data FIFO 570. The data FIFO 570 is shown in Figure 7 as resident in the TX 

22 logic 320. However, the data FIFO 570 may also be integral to the first level TX 

23 memory space 220. Data read transactions continue until all data to be transmitted 
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1 is stored in the memory subsystem 210 of the adapter 80. Following completion of 

2 the read operation, a status indication is returned to the LCP Client 1 00. Note that 

3 processing of the header can start as soon as the header has been read into the 

4 Headerin FIFO 400. There is no need to wait for the whole data to be read. 

5 E. Processing the frame header 

6 The header processing is performed by the TX processor 1 50. Header processing 

7 is protocol dependent and involves protocol information external to the LCP 

8 architecture. The TX processor 1 50 runs TX protocol header microcode and 

9 accesses routing tables and other relevant information already stored in the 

1 0 memory subsystem 2 1 0 of the adapter 80 during a protocol and routing 

1 1 initialization sequence. When the TX processor 1 50 receives an indication that a 

1 2 new header is waiting in the Headerin FIFO 400, it starts the header processing. 

1 3 The header processing produces one or more packet headers which are in the 

14 format employed to send packets over the network architecture 30 and include 

15 routing information. If the payload size is larger than a maximum packet size 

1 6 allowed by the network architecture 30, the payload is fragmented by generating 

17 several packet headers each used in connection with consecutive data segments of 

1 8 the original payload data to form packets for communication over the network 

19 architecture 30. 

20 F. Queuing the packet header for transmission 

2 1 A command defining the number of header words and the number of data words 

22 for a packet and the packet header itself are written by the TX processor 1 50 to a 

23 TX HeaderOut FIFO 540 in the first level memory space 220. 

24 G. Merging packet header and packet data for transmission. 
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Transmission of a packet on the network architecture 30 is triggered whenever a 
command is ready in the HeaderOut FIFO 540, and the data FIFO 570 contains 
enough data to complete the transmission of the related packet. A Cyclic 



4 



6 



5 



Redundancy Check (CRC) may be added to the header and data of each packet. 
Each complete packet is transferred to the network architecture 30 via the TX link 
logic 330. 



7 The transmission process for each frame is completed when all the frame data is 

8 transmitted on the network architecture 30, by means of one or more packets. For each 

9 frame processed by the adapter 80, a status may be returned to the application via a 

1 0 second LCP Client 1 00. This status indicates the completion of the frame data transfer 

1 1 from the host memory 60 onto the adapter 80, completion of the frame transmission itself, 

12 or other levels of transmission status. 

13 At any instance in time, the adapter 80 may be concurrently executing some or all of the 

14 following actions: selecting the next LCP to be serviced; initiating service for LCP 

1 5 channel A; executing DMA fetch of data for the last frame of LCP channel B; processing 

1 6 a frame header and fragmentation for LCP channel C ; and, transmitting packets 

1 7 originated by LCP channel D. 

1 8 Referring to Figure 8, what follows now, by way of example only, is a description of a 

1 9 data frame reception by an application using an RX LCP port. The operation of the ISOC 

20 120 may vary depending on the type of protocol supported by the LCP. Handshaking 

2 1 between the application and the adapter 80 assumes a prior initialization performed by the 

22 LCP manager 1 30. The RX LCP engme 360 comprises LCP allocation logic 620, LCP 

23 Context logic 6 1 0, and DMA logic 630 all residing in the LCP domain 520. The RX 

24 processor 1 60, first level RX memory space 230, and RX logic 350 all straddle the 

25 boundary between the frame domain 520 and the network domain 530. The RX link logic 
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1 340 and packet assist logic 600 reside in the network domain 530. In particularly 

2 preferred embodiments of the present invention, the HeaderOut FIFO 410 is located in 

3 the first level RX memory space 230. Frames received by the ISOC 120 from the network 

4 architecture 30 are written into LCP client buffers in the host memory 60. Availability of 

5 memory buffers is determined by the LCP RX client 100 and is indicated to the adapter 

6 80 for insertion of incoming data frames. The LCP client 100 provides buffers by writing 

7 into a receive Doorbell on the ISOC 120, similar to the aforementioned manner in which 

8 the transmission path logic 280 is informed of new frames ready to be transmitted. The 

9 Doorbell register address is allocated such that the write operation is translated into a 

10 physical write cycle on the bus architecture 70. The adapter 80 detects the write operation 

1 1 and logs the new provision of empty memory areas by incrementing the number of 

12 available word entries for the specific LCP RX Client 100. The available word count is 

13 part of the related LCP context 140. Whenever an application completes processing of a 

14 received frame within a buffer, it writes to the Doorbell. The write cycle indicates the 

1 5 number of words in the newly available memory space. The count within the LCP context 

16 is incremented by that amount. A packet received from the network architecture 30 may 

17 be part of a larger frame that will be assembled by the adapter 80 into contiguous space in 

18 the host memory 60. Processing of received frames by the ISOC 120 generally includes 

19 the following steps: 

20 A. Splitting packet header and data 

21 The RX link logic 340 translates information from the network architecture 30 

22 into a stream of packets. Each received packet is processed by the RX link logic 

23 340 to separate the packet header from the payload data. The header is pushed into 

24 an RX Headerin FIFO 640 in the first level RX memory space 230. The payload is 

25 pushed into an RX data FIFO 650 in the RX logic 350. The RX data FIFO 650 

26 may also be implemented in the first level RX memory space 230. 
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1 B. Decoding the packet header and generating and LCP frame header. 

2 The packet header is decoded to provide fields indicative of an ID for the frame to 

3 which the packet belongs, the size of the payload, and the size of the frame data. 

4 Once the packet header is reader for the RX Headerhi FIFO 640, an indication is 

5 sent to the RX processor 1 60. The RX processor processes the packet header 

6 information and generates an LCP related command including information 

7 required to transfer the packet data. Such information includes packet address and 

8 length. At the end of the header processing, a descriptor, or a set of descriptors, 

9 are written to the LCP RX HeaderOut FIFO 410, and an indication is triggered. 

10 C. Transfer of data within the RX LCP Context. 

1 1 The descriptors are fetched from the RX HeaderOut FIFO 410 by the RX LCP 

12 engine 360, and then decoded. The descriptors include the LCP number, packet 

13 address, packet data length and the source address of the data to be transferred in 

14 the memory subsystem 210 of the adapter 80. The RX LCP engine 340 uses the 

1 5 LCP Context information to create a target physical address (or addresses if a 

16 page is crossed) to be written to in the host memory 60 and initiates DMA 

1 7 transfers to write the data. 

1 8 D. ISOC DMA transactions. 

19 The ISOC 120 aims to optimize transactions on the bus architecture 70 by 

20 selecting appropriate bus conmiands and performing longest possible bursts. 

21 At any instance in time, the adapter 80 may be concurrently executing some or all of the 

22 following: processing a buffer allocation for LCP channel X; initiating an inbound data 

23 write service for LCP channel A; executing a DMA store of data for LCP channel B; 
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1 processing a frame assembly of a packet destined for LCP channel C; and, receiving 

2 packets for LCP channel D. 

3 To minimize frame processing overhead on the RX processor 1 60 and TX processor 1 50, 

4 packet assist logic 600 comprises frame fragmentation logic, CRC and checksum 

5 calculation logic, and multicast processing logic. 

6 The data flow between both the TX and RX LCP engines 3 1 0 and 360 and the host 1 0 

7 will now be described in detail. Both TX and RX LCP ports use memory buffers for 

8 transferring data and descriptor structures that point to such memory buffers. The 

9 descriptor structures are used to administer data buffers between a data provider and a 

1 0 data consumer and to return empty memory buffers to be used by the data provider. The 

1 1 descriptors point to the memory buffers based on either physical or virtual addresses. 

12 TX LCP channels are responsible for data transfer from the host memory 60 into buffers 

13 of the ISOC 120. Other layers of logic are responsible for transferring data from buffers 

14 of the ISOC 120 into the network 30. RX LCP channels are responsible for transferring 

1 5 data received from the network 30 to the host memory 60. 

16 The TX and RX LCP engines 310 and 360 are capable off handling a relatively large 

1 7 number of LCP channels. Each LCP channel has a set of parameters containing all 

18 information specific thereto. The information comprises the configuration of the channel, 

19 current state and status. The LCP context 140 associated with a channel is set by the LCP 

20 manager 130 during initialization of the channel. During channel operation, the content of 

21 the LCP context 140 is updated only by the ISOC 120. The LCP contexts 140 are saved in 

22 a context table within the memory subsystem 210 of the adapter 80. Access to the LCP 

23 context 140 of an LCP channel is performed according to the LCP number. The LCP RX 

24 and TX channels use different LCP context structures. 
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1 Data buffers are pinned areas in the memory 60 of the host 10. Transmit buffers hold data 

2 that for transmission. The TX LCP engine 3 1 0 moves the data located in these buffers 

3 into internal buffers of the ISOC 120. Incoming data received from the network 30 is 

4 moved by the RX LCP engine 360 into buffers in the memory 60 of the host 1 0. 

5 Ownership of the buffers alternates between software in the host 10 and the ISOC 120. 

6 The order of events on LCP TX channels is as follows: 

7 A. Software in the host 10 prepares buffers with data to be transmitted in the memory 

8 60 of the host 10; 

9 B. The software notifies the ISOC 120 that data in the buffers is ready to be 

10 transmitted; 

1 1 C. The ISOC 1 20 reads the data from the buffers; and, 

1 2 D. The ISOC 1 20 identifies to the software in the host 1 0 the buffers that were read 

1 3 and can be reused by the software in the host 1 0 to transfer new data. 

14 The order of events on LCP RX channels is as follows: 

1 5 A. The software in the host 1 0 prepares buffers into which the ISOC 2 1 0 can write 

1 6 the received data; 

17 B. The software notifies the ISOC 1 20 that free buffers are ready in the memory 60 

18 of the host; 

19 C. The ISOC 1 20 writes the data to the buffers; and, 

20 D. The ISOC 1 20 identifies to the software in the host 1 0 the buffers that were filled 

21 with received data and can be processed by the software. 

22 When the software prepares buffers to be used by the ISOC 1 20, buffer information is 

23 tracked via doorbell registers. Information relating to buffers used by the ISOC 120 is 

24 returned to the software using a status update or through a completion queue. For TX 

25 LCP channels, the buffers include data and header information transferred by the TX LCP 
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1 engine 3 1 0 into the ISOC 1 20 and processed to become one or more packets for 

2 transmission on the network 30. The header is used by the TX processor 1 50 of the ISOC 

3 120 to generate the header of the packet to be transmitted on the network 30. For RX LCP 

4 channels, free buffers are assigned by the software in the host 10 to the adapter 80. The 

5 adapter 80 fills the buffers with the received packets. 

6 The descriptors have defined data structures known to both the ISOC 120 and software in 

7 the host 10. The software uses descriptors to transfer control information to the ISOC 

8 120. The control information may be in the form of a frame descriptor, a pointer 

9 descriptor, or a branch descriptor depending on desired fiinction. Descriptor logic in the 

10 software and in the ISOC 120 generate and modify the descriptors according to control 

1 1 measures to be taken. Such measures will be described shortly. A frame descriptor 

12 comprises a description of the packet (e.g.: data length, header length, etc.). A pointer 

1 3 descriptor comprises a description of a data location. A branch descriptor comprises 

14 description of the descriptor location (e.g.: link lists of descriptors). Information in the 

1 5 descriptors is used for control by the software in the host 1 0 of the data movement 

1 6 operations performed by the TX and RX LCP engines 3 1 0 and 360. The information used 

1 7 to process a frame to generate a TX packet header is located in the header of the frame. 

1 8 Referring to Figure 9 A, descriptors may be provided in a single table 700 with the LCP 

1 9 context 140 pointing to the head of the table 700. Referring to Figure 9B, descriptors may 

20 also be arranged in a structure of linked descriptor tables 720-740. Following LCP 

21 channel initialization, the LCP context 140 points to the head of the first descriptor table 

22 720 in the structure. Branch descriptors 750-770 are used to generate a linked list of 

23 tables 720-740 where a branch descriptor 750-770 at the end of a descriptor table 720-740 

24 points to the beginning of another table 720-0740. Referring back to Figure 9A, branch 

25 descriptors can also be used to generate a cyclic buffer where a branch descriptor 71 0 at 

26 the end of a table 700 points to the beginning of the same table 700. A cyclic buffer may 

27 also be used in the receive path. In this case, the LCP 140 context is initiated to point to 

28 the head of the buffer. The buffer is wrapped around when the ISOC 120 reaches its end. 
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1 The software in the host 1 0 can write the descriptors into the memory 60 in the host 1 0 

2 (for both the receive and the transmit paths) or into the memory 250 of the adapter 80 (for 

3 the transmit path only). Writing descriptors to the memory subsystem 210 of the adapter 

4 80 involves an I/O operation by the software in the host 1 0 and occupies the memory 

5 subsystem 210 of the adapter 80. Writing descriptors in the memory 60 of the host 80 

6 requires the adapter 80 to access the memory 60 of the host 10 whenever it has to read a 

7 new descriptor. The location of the software descriptors is defined by the LCP manager 

8 1 30 for each LCP channel independently. The location of the descriptors is defined 

9 according to system performance optimization. The descriptors provide flexibility in the 

1 0 construction of queues. 

1 1 The RX and TX LCP engines 3 10 and 360 use addresses to access the descriptors in the 

12 descriptor tables and to access data buffers. An address can be either a physical address or 

13 a virtual address. The term physical address describes an address that the ISOC 120 can 

14 drive, as is, to the bus 70. The term virtual address describes an address which is not a 

15 physical one and is used by the software or microcode. The virtual address has to pass 

16 through a mapping in order to generate the physical address. An address used by the TX 

17 and RX LCP engines 310 and 360 can have different sources as follows: pointer in the 

1 8 LCP channel context 1 40; pointer in descriptors prepared by software running on the host 

19 10; pointer in descriptors prepared by the RX processor 1 60; and, pointer in descriptors 

20 prepared by the TX processor 1 50 (used for returning a completion message). A pointer 

2 1 can point to a descriptor or to a data buffer. Every address used by the TX and RX LCP 

22 engines 3 1 0 and 360 can be optionally mapped to a new address used as the physical 

23 address on the bus 70. The address mapping is done by the TX and RX LCP engines 3 1 0 

24 and 360. The ISOC 120 uses local memory 210 to hold the translation tables. The LCP 

25 manager 1 30 writes the translation tables to the adapter 80 during memory registration. 

26 The address mapping allows virtual addressing to be used for buffers or descriptor tables. 

27 The virtual addressing enables the management of virtual buffers that are physically 

28 located in more than one physical page. The address mapping also allows the host 1 0 to 
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1 work directly with applications using virtual addresses without requiring a translation 

2 processor for the software. 

3 Referring to Figure 10, shown therein is an image 800 of a buffer 880 as it appears to the 

4 software in the host 1 0. Also shown is a physical mapping 8 1 0 of the address at it is used 

5 to access the memory 60 in the host 10. A virtual pointer points 820 to a location in the 

6 buffer. The buffer in this example is a virtual buffer occupying a few noncontiguous 

7 pages 840-870 in the memory 60 of the host 10. The LCP engines 310 and 360 perform 

8 the mapping by translating the address via a translation table 830. The translation table 

9 holds a physical address pointer to the head of each physical buffer 840-870 mapped from 

1 0 the virtual buffer 880. Address mapping in the adapter 80 allows flexibility when 

1 1 mapping descriptors and data buffers in the memory 60 in the host 1 0. Address mapping 

12 in the adapter 80 also allows a direct connection to software buffers that use virtual 

1 3 addresses without requiring the software in the host 1 0 to perform address translation to a 

14 physical address. 

1 5 Each packet which the adapter 80 writes to the memory 60 in the host has a status 

16 associated therewith. The status allows synchronization between the adapter 80 and the 

1 7 software in the host 1 0. The status can be used to indicate different reliability levels of 

1 8 packets. The ISOC 1 20 provides the following status write backs: Transmit DMA 

1 9 Completion indicates that a data in a TX packet has been read into the adapter 80; 

20 Reliable Transmission is returned to indicate the completion of data transmission in the 

21 network 30; Receive DMA Completion indicates completion of a receive data transfer 

22 into the memory 60; and, Reliable Reception indicates reception of a transmit packet by a 

23 destination node in the network 30. 

24 A TX frame descriptor includes a 2 byte status field. Status write back means that a 

25 transaction status is written back into a descriptor. The status includes a completion bit 

26 which can be polled by the software in the host 1 0. When the software in the host 1 0 
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1 finds a set completion bit, it may reuse the buffers associated with the frame defined by 

2 the frame descriptor. 

3 A completion queue is implemented by an RX LCP channel. The LCP channel used by 

4 the completion queue has all the flexibility and properties that can be implemented by any 

5 RX LCP channel. The TX and RX processor 1 50 and 1 60 generates status write backs to 

6 indicate reliable transmission, reliable reception, receive DMA completion, or transmit 

7 DMA completion. Different indications relating to the frame are used in different cases. 

8 For example, in the case of a reliable transmission, the TX processor 1 50. Reads internal 

9 registers indicating the status of a packet transmission. In the case of reliable reception, 

10 the RX processor 160 gets a completion indication as a received packet which includes an 

1 1 acknowledgment. In the case of a receive DMA completion, the RX processor 1 60 uses 

12 frame completion information. In the case of a transmit DMA completion, the TX 

13 processor 150 indicates the reception of a frame for transmission in the adapter 80. A 

14 completion queue can be used by a single TX or RX LCP channel or may shared by 

15 multiple channels. Micro code in the adapter 80 updates a status queue by initiating a 

16 frame descriptor into a command queue of the RX LCP engine 360. Referring to Figure 

17 1 1 , the status is transferred to the memory 60 of the host 1 0 via a completion status LCP 

1 8 900 comprising a completion queue 920. The completion queue 900 is continuous (either 

1 9 physically or virtually) and is located in the memory 60 of the host 1 0. For example, the 

20 completion queue can be held in a continuous buffer. Entries 930 in the completion queue 

21 preferably have a fixed size. Each entry holds a pointer 940 to the head of a buffer 950 

22 associated with a receive LCP 910. The buffer 950 is filled by the packet 960 associated 

23 with the completion status. . 

24 A TX software/adapter handshake comprises an TX LCP port and an completion RX LCP 

25 port. Each LCP transmit channel uses the following data structures: 



D cket Number IL920000077US1 



-29- 



1 A Doorbell entry, implemented as a memory mapped address, informs the adapter 

2 80 of incremental requests to process descriptors and data. Each process has a 

3 unique access into a single page of memory mapped address used for Doorbell 

4 access. 

5 An LCP context entry in the adapter memory space 2 1 0, containing LCP attributes 

6 and status fields. 

7 A structure of transmit descriptors. This structure may span across multiple 

8 physical pages in the memory 60 of the host 10. If virtual addressing is used for 

9 the descriptors, a translation table is used to move one page to the next. If physical 

10 addressing is used for the descriptors, branch descriptors are used to move fi*om 

1 1 one page to the next. Transmit descriptors contain a status field that can be 

12 updated following transfer of all descriptor related data to the adapter 80. 

1 3 Transmit data buffers pinned in the memory 60 of the host 1 0 pointed to by the 

14 pointer descriptors. If virtual addressing is used for the data buffers, a translation 

1 5 tale converts the pointer into physical addresses used by the adapter 80 to access 

1 6 the memory 60 in the host 1 0. 

1 7 A translation table and protection blocks in the adapter memory space 2 1 0 are 

1 8 used for address mapping. 

19 Referring to Figure 12, a transmit packet flow comprises, at step 1000, software 1020 in 

20 the host 10 filling buffer 1030 with data to be transmitted. At step 1010, the software 

2 1 1 020 updates the descriptors 1 040. The descriptors 1 040 may be either in the memory 60 

22 of the host 1 0 or in the memory subsystem 2 1 0 of the adapter 80. At step 1 050, the 

23 software 1 020 rings the Doorbell to notify the adapter 80 that new data is ready to be 

24 transmitted. At step 1060, the adapter 80 manages arbitration between requests fi-om the 
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1 different LCP channels. When a channel wins the arbitration, the adapter 80 reads the 

2 new descriptors 1040. At step 1070, the adapter 80 reads the data. At step 1080, the data 

3 is transmitted to the network 30. At step 1090, the status is updated in the descriptors 

4 1 040 or in the completion queue. 

5 The TX LCP channel may use address translation when accessing data buffers. In this 

6 case, the data buffer is composed of multiple memory pages. As far as the process is 

7 concerned, these memory pages are in consecutive virtual memory space. However, as far 

8 as the adapter 80 is concerned, these memory pages may be in nonconsecutive physical 

9 memory space. A completion status structure contains information indicative of the status 

10 of transmitted frames. This is implemented as a separate LCP channel The frame 

1 1 descriptor, which is the first descriptor for every fame, has an optional status field which 

12 can be updated after the frame has been transferred to the adapter 80. 

1 3 Referring now to Figure 1 3, in an example of a transmit LCP channel flow, descriptors 

14 1 1 00 are located in the memory 60 of the host 1 0. Access to the descriptors 1 11 0 and 

1 5 buffers 1110 storing packets 1 1 20 requires address translation through a translation table 

16 1130 located in the adapter 80. The buffers 1 1 1 0 use contiguous space in the virtual 

17 address space of the software in the host 10. Each frame 1 120 is described by two types 

18 of descriptors: a frame descriptor 1 1 40 giving information relating the packet; and, a 

1 9 pointer descriptor 1 1 50 pointing to the buffer 1110 holding the data 1 1 20. Each packet 

20 comprises a data payload 1 1 70 preceded by a header 1 1 60 in the same buffer 1 1 80. 

2 1 A write transaction 1 1 90 to the Doorbell updates the nimiber of words 1 200 available for 

22 use by the adapter 80. This information is stored in the LCP context 140. The transmit 

23 LCP context 140 includes a pointer 1210 to the head of the buffer 1110 holding the data 

24 to be transmitted. When the LCP channel wins the internal channel arbitration of the 

25 ISOC 120, the ISOC 120 reads the descriptors of the LCP channel according to the 

26 pointer 1210 in the LCP context 140. Virtual addresses, for both descriptors 1 100 and 
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1 buffers 1 1 10 of the LCP channel, are translated into physical addresses using the 

2 translation table 1130 located in the memory subsystem 2 1 0 of the adapter 80. The 

3 translation table 1 1 30 is updated by the LCP manager 140 during registration of the 

4 memory buffers. The ISOC 1 20 reads the data and frame headers from the buffers 1110 

5 into the adapter 80. The frame headers 1 160 are then replaced on the ISOC 1320 by a 

6 header for the network 30. The packet header and the corresponding data are then 

7 transmitted to the network 30. 

8 The RX LCP port is used to transfer incoming data from the ISOC 1 20 to the memory 60 

9 used by a software application running on the host 10. TX LCP channels are completely 

10 controlled through descriptors initiated by the software on the host 1 0. RX LCP chaimels 

1 1 use descriptors from both the software on the host 1 0 and the ISOC 120. The descriptors 

12 initiated by the ISOC 120 are used to control the LCP channel operation to define the 

1 3 destination of a received frame in the memory 60 of the host 1 0. The descriptors initiated 

1 4 by the software in the host 1 0 can be used to define the location of buffers where the 

15 buffers were not defined through mapping in a translation table. To implement a 

16 handshake between the software in the host 10 and the adapter 80, two LCP channels are 

1 7 preferably used: an RX LCP channel for handling the received incoming data structure; 

1 8 and, an RX LCP chaimel for handling the completion status queue. The completion status 

1 9 is used by the adapter 80 to signal to the software in the host 1 0 that a frame transfer into 

20 the memory 60 of the host 10 is completed. Entries are inserted into the completion queue 

21 structure in sequential addresses. Each completion status entry contains a field that is 

22 marked by the adapter 80 and pooled by the software in the host 1 0 to check that the entry 

23 ownership has been transferred from the adapter 80 to the software in the host 10. One or 

24 more RX LCP channels can use the same completion status queue. The sharing of the 

25 completion status queue by muUiple RX LCP channels is performed by the ISOC 120. 
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1 An RX LCP channel requires information to indicate the destination address for an 

2 incoming packet. The ISOC 120 has two addressing for finding the location of free 

3 buffers: 

4 Direct addressing mode refers to LCP channels that do not use pointer descriptors 

5 to point out a buffer. The destination address is defined either by microcode in the 

6 ISOC 1 20 or read from the context 1 40. 

7 Indirect addressing mode refers to LCP channels that maintain pointers to data 

8 buffers in descriptor structures. The descriptors are preferably located in the 

9 memory 60 of the host 1 0. 

1 0 Direct addressing substantially cuts down the latency of processing an incoming packet 

1 1 through the adapter 80. However, it requires registration of memory buffer by the LCP 

12 manager 1 30, including storage of virtual to physical translation information on the 

13 adapter 80. The software in the host 10 writes to the channels Doorbell to indicate the 

14 amount of words added to the free buffer that can be used by the channel In direct mode, 

1 5 the following steps are used to determine the address of the destination buffer: 

16 A. Address A is driven as a command to the LCP engine. 

17 B. (Optional) Address A is mapped to address A' . 

1 8 C. Address A' (if step B is executed) or A (if step B is not executed) is the base 

1 9 address for the destination buffer. 

20 In indirect mode, the adapter 80 uses descriptors to find the address of the data buffers. 

2 1 The descriptors are managed by the software in the host 1 0. The descriptors are preferably 

22 located in the memory 60 of the host 10. The term indirect is used to emphasize that the 

23 adapter 80 reads additional information to define the destination address. The adapter 80 

24 accesses this information during run-time. Indirect addressing cuts down the amount of 
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2 
3 



the memory n the adapter 80 required to store translation tables. The descriptors are 
typically located in the memory 60 of the host 10. In indirect mode, the following steps 
are used to determine the address of the destination buffer: 



4 


A. 


Address A is driven as a command to the LCP engine. 


5 


B. 


(Optional) Address A is mapped to address A'. 


6 


C 


Address A' (if step B is executed) or A (if step B is not executed) is the address of 


7 




the pointer descriptor. 


8 


D. 


The pointer to the buffer, address B, is read from the descriptor. 


9 


E. 


(Optional) Address B is mapped to address B'. 


10 


F. 


Address B' (if step E is executed) or B (if step E is not executed) is the base 


11 




address for the destination buffer. 


12 


Each RX LCP channel uses the following data structures: 


13 




Access to the Doorbell, implemented as a memory mapped address, informs the 


14 




adapter 80 of additional data or descriptors available for the adapter 80 to write 


15 




packet data. 


16 




An LCP context entry in the memory space 210 of the adapter 80 contains LCP 


17 

1 / 




attributes, state, configuration, and status fields. 


18 




Descriptors pointing to memory buffers for use in indirect mode. 


19 




A buffer in contiguous virtual address space in the memory 60 of the host 10. 


20 




A translation table and protection blocks in the memory space 210 of the adapter 


21 




80 for address mapping. 


22 


The flow of receiving a packet depends on the following characteristics: 


23 




Direct or indirect addressing mode. 


24 




For indirect mode, descriptors are located in the memory 60 of the host 10. 
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5 



For direct mode, address mapping may or may not be used during access to 
descriptors. 

Address mapping may or may not be used during access to buffers. 

For indirect mode, address protection may or may not be used during access to 

descriptors. 

Address protection may or may not be used during access to buffers. 



7 These characteristics are set for each LCP channel as part of the channel's context 140 

8 during the LCP channel initialization. 

9 Referring to Figure 14, a flow of receive packets comprises, at step 1300, preparation by 

10 software 1310 in the host 10 of free buffer 1320 for the received data. At step 1330, in 

1 1 indirect mode, the software 1 3 1 0 in the host 1 0 updates the descriptors 1 340. The 

12 descriptors 1340 are located in the memory 60 of the host 10. At step 1350, the software 

13 in the host 10 rings the Doorbell to notify the adapter 80 of the free buffer space. For 

14 indirect mode, the Doorbell provides information indicative of the new descriptors 1340. 

15 For direct mode, the Doorbell provides information indicative of added free buffer space. 

16 At this stage, the adapter 80 is ready to transfer receive data from the network 30 to the 

17 memory 60 of the host 10. Steps 1300, 1330, and 1350 are repeated whenever the 

1 8 software 1 3 1 0 in the host 1 0 adds free buffers 1 320 to the RX LCP channel. The ISOC 

19 120 repeats the following steps for each received packet. At step 1360, the adapter 80 

20 receive the data. At step 1 370, in indirect mode, the adapter 80 reads descriptors 1 340 

21 pointing to the location of the free data buffers 1320. At step 1380, data and headers are 

22 written into the data buffers 1 340. At step 1 390, status is updated in the completion 

23 queue. 

24 Referring to Figure 1 5, in an example of a receive LCP channel flow, pointer descriptors 

25 are not used. Furthermore, no translation tables are used. Data buffers 1400 use 

26 contiguous space in the physical address space of software in the host 10 using the buffers 
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1 1400. Both header and data payload are written to the buffers 1400. A write transaction 

2 1410 to the Doorbell updates the data space available for use by the adapter 80. The 

3 information is stored in the LCP context 140. The receive/completion LCP context 140 

4 includes a pointer 1420 to the head of the buffer 1400 and an offset 1430 to the 

5 next/current address used to write new data/completion entries. When the adapter 980 

6 receives a packet, it increments the offset 1430 to the next packet location and updates the 

7 available data space. A completion entry 1440 is added to a completion LCP 1450 upon 

8 completion of a frame reception, upon frame time-out, or for any other frame event that 

9 requires awareness from the LCP client 100. The completion entry 1440 contains all the 

1 0 information needed by the LCP client 1 00 to locate the frame within the LCP data buffer 

1 1 1400. The software in the host 10 uses a field within the completion entry 1440 to 

12 recognize that it has been granted ownership of the completion entry 1440. 

1 3 The ISOC 1 20 allows LCP channels to be used for moving data between the memory 

14 subsystem 210 of the adapter 80 and the memory 60 of the host 10. To transfer data from 

15 the memory 60 of the host 10 to the adapter 80 a transmit channel is used. To transfer 

16 data from the adapter 80 to the memory 60 of the host 10 a receive channel is used. When 

1 7 data is to be transferred from the memory 60 of the host 1 0 to the adapter 80 a frame 

18 descriptor includes a destination address on the bus 340 of the ISOC 120. This address 

19 defines the destination of the frame data payload. The packet header is transferred in the 

20 usual manner. This allows loading of tables and code into the memory space of the ISOC 

21 120. To transfer data from the memory space of the ISOC 120 to the memory 60 of the 

22 host 10 using a receive channel a descriptor is initiated by the RX processor 160. The 

23 descriptor include information indicative of both destination address in the memory 60 of 

24 the host 10 and source address. 

25 In preferred embodiments of the present invention hereinbefore described, the adapter 80 

26 is connected to the CPU 50 and memory 60 of the host computer system 1 0 via the bus 

27 architecture 70. However, in other embodiments of the present invention, the adapter 80 



Docket Number IL920000077US1 



-36- 



1 may be integrated into the host computer system 10 independently of the bus architecture 

2 70. For example, in other embodiment of the present invention, the adapter 80 may be 

3 integrated into the host computer system via a memory controller connected to the host 

4 memory 60. 

5 Additionally, in preferred embodiments of the present invention hereinbefore described, 

6 the adapter 80 was implemented in the form of a pluggable adapter card for insertion into 

7 the host computer system 1 0. It will however be appreciated that different implementation 

8 of the adapter 80 are possible in other embodiments of the present invention. For 

9 example, the adapter 80 may be located on a mother board of the host computer system, 

1 0 along with the CPU 50 and the memory 60. 

1 1 Variations described for the present invention can be realized in any combination 

12 desirable for each particular application. Thus particular limitations, and/or embodiment 

1 3 enhancements described herein, which may have particular advantages to a particular 

14 application need not be used for all applications. Also, not all limitations need be 

1 5 implemented in methods, systems and/or apparatus including one or more concepts of the 

1 6 present invention. 

1 7 The present invention can be realized in hardware, software, or a combination of 

1 8 hardware and software. A visualization tool according to the present invention can be 

19 realized in a centralized fashion in one computer system, or in a distributed fashion where 

20 different elements are spread across several interconnected computer systems. Any kind 

21 of computer system - or other apparatus adapted for carrying out the methods and/or 

22 fimctions described herein - is suitable. A typical combination of hardware and software 

23 could be a general purpose computer system with a computer program that, when being 

24 loaded and executed, controls the computer system such that it carries out the methods 

25 described herein. The present invention can also be embedded in a computer program 

26 product, which comprises all the features enabling the implementation of the methods 
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1 described herein, and which - when loaded in a computer system - is able to carry out 

2 these methods. 

3 Computer program means or computer program in the present context include any 

4 expression, in any language, code or notation, of a set of instructions intended to cause a 

5 system having an information processing capability to perform a particular function either 

6 directly or after conversion to another language, code or notation, and/or reproduction in 

7 a different material form. 

8 Thus the invention includes an article of manufacture which comprises a computer usable 

9 medium having computer readable program code means embodied therein for causing a 

1 0 function described above. The computer readable program code means in the article of 

1 1 manufacture comprises computer readable program code means for causing a computer to 

12 effect the steps of a method of this invention. Similarly, the present invention may be 

1 3 implemented as a computer program product comprising a computer usable medium 

14 having computer readable program code means embodied therein for causing a a function 

1 5 described above. The computer readable program code means in the computer program 

16 product comprising computer readable program code means for causing a computer to 

1 7 effect one or more functions of this invention. Furthermore, the present invention may be 

1 8 implemented as a program storage device readable by machine, tangibly embodying a 

19 program of instructions executable by the machine to perform method steps for causing 

20 one or more fiinctions of this invention. 

21 It is noted that the foregoing has outlined some of the more pertinent objects and 

22 embodiments of the present invention. This invention may be used for many 

23 applications. Thus, although the description is made for particular arrangements and 

24 methods, the intent and concept of the invention is suitable and applicable to other 

25 arrangements and applications. It will be clear to those skilled in the art that 

26 modifications to the disclosed embodiments can be effected without departing from the 
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1 spirit and scope of the invention. The described embodiments ought to be construed to 

2 be merely illustrative of some of the more prominent features and applications of the 

3 invention. Other beneficial results can be realized by applying the disclosed invention in 

4 a different manner or modifying the invention in ways known to those familiar with the 

5 art. 
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